Multimodal literacy in a new era of educational technology : Comparing points of view in animations of children's and adult literature

Purpose The paper shows the interpretive impact of different constructions of the point of view available to the reader/viewer in book and animated movie versions of a children's picture book, a novel for pre-adolescents/early teenagers, and a graphic novel for adolescents and adults. Design/Approach/Methods Excerpts from book and animated movie versions of the same story are compared using multimodal analysis of interpersonal meaning to show how the reader/viewer is positioned in relation to the characters in each version, complemented by analyses of ideational meaning to show the effect of point of view on interpretive possibilities. Findings Focusing mainly on multimodal construction of point of view, the analyses show how interpretive possibilities of ostensibly the same story are significantly reconfigured in animated adaptations compared with book versions even when the verbal narrative remains substantially unchanged. Originality/Value The study shows that it is crucial to students’ critical appreciation of, and their creative contribution to, their evolving digital literary culture that in this new era of educational technology, attention in literacy and literary education focuses on developing understandings of digital multimodal narrative art, and that animated movie adaptations are not presented pedagogically as isomorphic with, or simply adjunct to, corresponding book versions

The Logogenesis of Writing to Learn: A Systemic Functional Perspective.

Writing to learn has become an important practice in science education. How is scientific knowledge constructed during writing? To investigate this question, we examined the process through which four university students constructed written explanations of either projectile motion or buoyancy. The analysis, informed by systemic functional linguistics, focused on the mapping of semantic elements to grammatical choices, and the way in which this mapping unfolded throughout the course of each text. The texts began largely congruently; grammar mapped closely to experience. Gradually, each text shifted toward greater use of grammatical metaphor. Nominalization allowed propositions and sequences of events to serve as participants in complex causal and epistemic relationships. Students’ texts showed several properties of professional scientific texts: transcategorization, compaction, and logicality; however, professional science texts instantiate these properties synoptically and systemically, whereas student texts exemplify them dynamically and instantially

Reworking the appraisal framework in ESL research : Refining attitude resources

The Appraisal framework within Systemic Functional Linguistics as a robust tool in language teaching and research has attracted a great deal of interest in recent years. Since its establishment as the most complete account, the framework has been used in a variety of contexts, resulting in a number of refinements, tuning its applicability for specific research purposes such as studies of the evaluative language in research article abstracts, biology experiment reports, wine appreciation and student narrative writing. This article proposes additional refinements, particularly to the system of Attitude, informed by research into the deployment of evaluative resources in spoken discourse by postgraduate students in small group discussions in English and in Vietnamese. The refinements were required to account for the range of evaluative language used in discussions of topics including personal experiences of living and studying in Australia, academic experiences at Australian universities, and opinion about one’s professional standing. These refinements contribute to the ongoing development of the Appraisal framework and provide a resource for enhancing the effectiveness of expressions of evaluative stance for speakers of English as a second or additional language

Including visual representations within senior high school biology assessment : Considerations of grammatical complexity

This paper analyses the opportunities for presenting knowledge that are created when assessment allows senior high school biology students to draw on linguistic and visual resources when constructing meaning in response to short-answer examination-style questions requiring a sequential explanation. Students within one senior high school biology class were given the opportunity to respond to an examination-style question through both written and visual representations. Analysis of the student responses for high and middle-achieving students, from a systemic functional linguistics perspective, indicates that high-achieving students use a broader range of grammatical forms more often than middle-achieving students to present key understandings of classification and composition within both written and visual representations. Including opportunities within assessment for students to express knowledge through written and visual representations allows for students to elaborate within their short-answer responses and to construct the broader range of representations that is valued within the discipline, but explicit guidance is required to support all students to make use of the complex grammatical patterns within written and visual representation. For senior high school biology students to be successful in the final stages of schooling, explicitness about the complex grammars of visual and written representations is required within curriculum and pedagogy

The multimodal construction of race : A review of critical race theory research

Issues of race periodically rupture in the national and internationalconsciousness, while at other times, there is a false belief thatsociety has arrived at a post-racial era. Either way, there remainsimpetus for the critical interrogation of the racialisation ofmultimodal literacies in education, and critical race theory (CRT) is aleading approach. This article reviews original studies thatcollectively analyse multimodal texts and practices to understandthe construction of race in education. Multimodal texts haveproliferated in online textual ecologies due to the ease ofproduction and rapid global dissemination of image-based texts inthe twenty-first century. Such texts combine two or more modes,such as images, words, sounds, and gestures. Sites for thecirculation of multimodal literacies–online and offline–serve todisrupt, reify, or perhaps even exacerbate racial identities, prejudice,and subordination in education. The review highlights the prevalentthemes: (a) Discursive construction of race in the spoken mode, (b)Anti-racist and multimodal counter-narratives, (c) The racialisationof multimodal literature for children and adolescents, and (d) Racein music, visual and performing arts, and digital media. Gaps in CRTresearch and challenges are posed for future research of race in thecontext of cultural and technological change

Multimodal literacy

Multimodal literacy is a term that originates in social semiotics, and refers to the study of language that combines two or more modes of meaning. The related term, multimodality, refers to the constitution of multiple modes in semiosis or meaning making. Modes are defined differently across schools of thought, and the classification of modes is somewhat contested. However, from a social semiotic approach, modes are the socially and culturally shaped resources or semiotic structure for making meaning. Specific examples of modes from a social semiotic perspective include speech, gesture, written language, music, mathematical notation, drawings, photographic images, or moving digital images. Language and literacy practices have always been multimodal, because communication requires attending to diverse kinds of meanings, whether of spoken or written words, visual images, gestures, posture, movement, sound, or silence. Yet, undeniably, the affordances of people-driven digital media and textual production have given rise to an exponential increase in the circulation of multimodal texts in networked digital environments. Multimodal text production has become a central part of everyday life for many people throughout the life course, and across cultures and societies. This has been enabled by the ease of producing and sharing digital images, music, video games, apps, and other digital media via the Internet and mobile technologies. The increasing significance of multimodal literacy for communication has led to a growing body of research and theory to address the differing potentials of modes and their intermodality for making meaning. The study of multimodal literacy learning in schools and society is an emergent field of research, which begins with the important recognition that reading and writing are rarely practiced as discrete skills, but are intimately connected to the use of multimodal texts, often in digital contexts of use. The implications of multimodal literacy for pedagogy, curriculum, and assessment in education is an expanding field of multimodal research. In addition, there is a growing attention to multimodal literacy practices that are practiced in informal social contexts, from early childhood to adolescence and adulthood, such as in homes, recreational sites, communities, and workplaces

The effect of unknown words and the semantic structure of texts on reading comprehension

The correlation between word knowledge and reading comprehension is well established but the nature of the relationship remains unclear. It involves complex interactions among factors such as the proportion of unfamiliar words in the passage, the extent of redundancy associated with these words, their relative salience to the semantic organization of the text and the readers‘ proficiency in responding to these text characteristics. This study is one of the few which have investigated these interrelationships. It sought to determine whether good readers differed from poor readers in utilizing redundancy in text to construct meanings for unknown words and whether good readers and poor readers were more likely to so use redundancy when unknown words were salient to the story line rather than peripheral. The study further investigated how the degree of salience and redundancy of unknown words affected recall of the gist of a story and the total story information recalled by good and poor readers. An initial test of fifth grade pupils' recall of the gist of an aural story was used to eliminate from the study those pupils whose scores indicated an inadequately developed schema for simple stories. This test revealed a significant difference between good and poor readers (categorized according to standardized reading test scores) and wide variation among the scores of poor readers, indicating support for previous research which suggested that certain subgroups of poor readers had inadequately developed story schemata. Data from pupils whose aural recall scores did not reach criterion were not included in the subsequent analyses. The good and poor readers later read experimental versions of two stories containing six percent unknown words of varying redundancy and salience. Tests of the meanings of these words revealed redundancy effects for only one story. In this story good readers and poor readers used redundancy to construct meanings for unknown words but good readers scored higher and poor readers did not utilize redundancy when the unknown words were salient. This was attributed to the increased processing demands of salient compared with peripheral unknown words and the greater flexibility of good readers in coordinating multiple processing goals in reading. Implications for the teaching of context clues were discussed. Recall scores for one story only were relevant since, in the other story, good readers proved to be familiar with target words which were assumed to be unknown. For the story which did contain unknown words, good readers' gist and total recall were impeded only when unknown words were salient and not redundant. Although poor readers recalled less of the gist and total story information, their pattern of gist recall was somewhat similar to that of good readers but, by contrast, the relative salience and redundancy of unknown words had no differential effects on poor readers' total recall scores. Implications for assessment of reading based on miscue analysis procedures were discussed

New challenges for literature study in primary school English : Building teacher knowledge and know-how through systemic functional theory

Australian primary school teachers face two major challenges in their implementation of the national curriculum for English: literary study and multimodality. Whilst teachers and students frequently engage with texts like literary picture books, the requirement that teachers build children’s understandings of texts as patterned, aesthetic constructs is new. And it is especially demanding for teachers without specialized training in either literature or multimodality. They must learn to manage the expanded ‘reservoir’ of meaning in school English and develop ‘repertoires’ of semiotic understanding in the course of fulltime teaching (Bernstein, 2000). This paper emerges from a larger study that aimed to meet the challenge of literary study in English by introducing practicing teachers to a semiotic toolkit inspired by systemic functional grammatics. Grammatics, as Halliday (2002) interprets it, distinguishes the theory from the practice of grammar, the metalanguage from language in use. In our project, systemic functional grammatics included study not just of clause-level choices in language but their role in larger discourse frames and, via analogy, in images and multimodal texts. We made use of the ‘resemblance’ between focalization in print narratives and in bi-modal narratives picture books. Adapting semiotic principles like stratification and metafunction to national curriculum notions of ‘levels of analysis’ and ‘threads of meaning’, we used systemic functional (SF) theory to open up the potential of literature study for English teachers in NSW and Victoria, attempting to build understanding about the ‘uses’ of grammatics for a relatively uninformed group of ‘users’ (Martin et al., 2013). Because of the need to manage the theory-practice nexus in professional learning, we attempted to characterize ‘knowledge about’ images in narrative in accessible and systematic ways and to relate this to pedagogic ‘know-how’ in primary teaching and assessment of narrative. The paper introduces the analytical framework we developed to represent and develop knowledge and know-how in primary school literature study. It shows how we used the framework to benchmark teacher starting-points as they commented on students’ responses to a picture book called The Great Bear by Armin Greder and Libby Gleeson (1999). It overviews input provided to teachers in workshops based on SF principles such as system, stratification and metafunctions. Finally, it overviews our initial findings based on our analysis of follow-up interviews with two teachers as they reflected on students’ responses to The Tunnel, by Anthony Browne (1989). Changes are arrayed on clines produced to account for shifts in teacher knowledge and know-how. Early results of our project are very encouraging, providing evidence of significant if varied growth in teachers’ orientations to narrative meaning and increased levels of meta-semiotic awareness. The paper concludes with reflections on the use of SF grammatics for meeting the challenges of literature study in primary school English in an era of multimodality

Enhancing expressions of attitudes: Achieving equity for international students in everyday communication

International students from language backgrounds other than English (LBOTE) often have problems expressing attitudes precisely and effectively in English. Not being able to express what they think and feel prevents international students from adjusting easily to the host culture and integrating socially with the locals, including local students. Current ELICOS programs often neglect attitudinal language. This article suggests one way of enhancing international students’ fluency and confidence in everyday communication through analysis of the language used in a narrative extract. Teaching and learning activities focus on scaffolding students’ proficiency in using evaluative language (Martin & White, 2005)